A Survey of Corpora in Computational and Cognitive Narrative Science

The use of corpora, and especially annotated corpora, is key to advancing the scientific understanding of narrative. I seek here to answer several interrelated questions: what corpora have been used in computational and cognitive research on narrative to date? What are the general characteristics of these corpora, and are they adequate for the overall task of a scientific understanding of narrative? What types of narratives have been studied, what kinds of annotations have been applied to those narratives, and what gaps need to be filled? To answer these questions, I conducted a wide-ranging literature review, collecting a bibliography of over 2600 conference papers, journal articles, theses, reports, and books on the topic of the computational, cognitive, and (more generally) scientific1 study of narrative. I surveyed this bibliography for the use of corpora, and identified 167 unique text collections (155 with some sort of annotation) that could be considered a “corpus”. These corpora were used in more than 180 publications across twelve fields. This set contained seventeen different broad types of narratives, five different modalities, and approximately 42 different types of annotation. Of the types of annotations present, the most common complex is an annotation of Events, Named Entities, and Roles (~33 % of corpora) which indicates that a representation of the “who does what to whom” (i.e., the fabula) of narratives has been a key concern of researchers. Worryingly, I was able to identify only four corpora which provide enough information to capture a basic representation of fabula that were not annotated directly by their authors. This means that there is much work to be done in improving data collection standards in the field. Another observation was that the sense and meaning of words, narrative units, and other levels of the narratives is woefully under-represented. Finally, another important finding is that over half of the corpora identified have fewer than 20 texts, and many of these “narrative” corpora contain texts that are only barely “narrative” in nature. These observations lead to some clear recommendations for the directions of future research and resource-collection efforts. The paper is organized as follows. First I discuss the need for and utility of corpora in the scientific study of narrative (§1). I then outline the method I used for surveying the literature (§2), which resulted in a comprehensive list of 64 workshops, symposia, conferences, and other meetings that have been held on narrative science in the past 30 years (Table 7 in the Appendix). I then proceed to a detailed analysis of the results of the survey, identifying the different types of narratives, modalities, annotations, annotation methods, and so forth that I found (§3). Because the type and extent of annotation is